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(57) Claims 

1 Control device for a manipulator system which comprises a master/slave type 
manipulator, a television camera, and a television monitor, wherein said manipulator 
control device comprises a control circuit that designates as position information for the 
slave a value which is proportional to a value obtained by dividing current position 
information for the master by the zoom factor of the television camera, and that makes 
manipulator displacement displayed on the television monitor substantially equal to 
displacement of the master regardless of zoom factor. 

2 Control device according to claim 1 comprising a circuit that designates force 
information, proportional to product of current force information for the slave and the 
zoom factor, as force instruction information for the master. 

3 Control device according to claim 1 or 2 comprising a switchover circuit that 
designates a parameter for invoking a master motor signal as the master and slave force 
signal in the event of a time change in zoom factor, or as position information for the 
master and slave in event that there is no time change in zoom factor. 

4 Control device according to claim 3 comprising a circuit that fixes the slave joint 
at its current position in the event that the time change in zoom factor is 0. 

Detailed Description of the Invention 

(Field of Utilization of the Invention) 

The present invention relates to a control device for a manipulator system for use 
in environments not easily approachable by humans (such as a nuclear force generating 
plant) and that is equipped with a television monitoring system enabling unmanned 
operation by remote control. 
(Background of the Invention) 

Methods for remote operation of master/slave manipulators have been disclosed 
inter alia in Unexamined Patent Application 59-24973 and Unexamined Patent 
Application 59-47174. Unexamined Patent Application 58-137575 teaches an example 
of control using a pair of television cameras to produce a three-dimensional image. In 
this example, a three-dimensional image of the slave manipulator and the master 



2 




manipulator may be viewed simultaneously superimposed at the same size. However, 
even a slight difference in size between the actual image of the master manipulator and 
the virtual image of the slave manipulator viewed by the operator v^U result in relative 
deviance, differing in a manner dependent on location, in displacement of the master 
manipulator and displacement of the slave manipulator, creating the problem of extreme 
difficulty in both observation and operation. 

Where the virtual image of the slave manipulator is placed apart to the firont of the 
actual image of the master manipulator in order to reduce this effect, not only is the image 
small, but it is also difficult to produce fine movement of the manipulator. The causes of 
this are backlash and friction in the force transmission mechanism of the manipulator, and 
failure to provide adequate visual information. 
(Object of the Invention) 

It is an object of the present invention to provide a manipulator control device for 
a manipulator that is maneuvered while viev^ng an image on a monitor television, 
whereby an operator may readily control extremely fine movement and slight force levels 
of the manipulator, and which is suitable for producing a realistic impression 
indistinguishable fi'om actual viewing of the slave manipulator. 
(Summary of the Invention) 

The present invention provides, in a system comprising television camera having 
a zoom lens; a television monitor; and a master/slave manipulator; a control circuit that 
expands or contracts motion of the master arm for transmission to the slave, in a manner 
dependent on the magnification M of the image shown on the television monitor. 
Displacement of the distal end of the slave appearing on the television screen and 
displacement of the distal end of the master arm are controlled by this circuit to constant 
values regardless of zoom factor. As a result, visual information presented to the operator 
regarding motion of the slave arm matches the movement of the arm as the operator 
manipulates the master arm, enhancing the perception of integration between operator 
and slave arm. In conventional systems, in zoom up, motions of the slave arm on the 
television screen are large relative to [actual motions of] the master arm, making 
manipulation difficult. According to the present invention, however, displacement of the 
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distal end of the slave appearing on the television screen and displacement of the distal 
end of the master arm are constant regardless of zoom factor, thus improving operability 
of the manipulator. 
(Embodiments of the Invention) 

An embodiment of the invention is described hereinbelovs^ with reference to the 
accompanying drawings. Fig. 2 shows a system design implementing the present 
invention. The system comprises a master manipulator la controlled by an operator 9; a 
slave manipulator lb of similar shape thereto, that operates at the actual work site; a 
television camera 6 for filming the work site; a television monitor 5 that allows the 
operator 9 to view the work site from a remote location; and a control device 15. Master 
manipulator la is suspended from a crossbar 2. Master manipulator 2 has a shoulder joint 
10a, elbow joint 1 la, and vmst joint 12a; the arm has a degree of freedom of 6 (directions 
of motion are 101a -106a). The slave manipulator lb similarly has a shoulder joint 10b, 
elbow joint 1 lb, and wrist joint 12b, and an arm degree of freedom of 6 (directions of 
motion are 101b -106b). Rotation angles of the joints of the two manipulators are 
controlled independently for each degree of freedom. That is, as indicated in Table 1 , the 
joint for the master base rotation direction 101a and the joint for the slave base rotation 
direction 101b constitute one set; the joint for the master shoulder rotation direction 102a 
and the joint for the slave shoulder rotation direction 102b constitute [another] set, and so 
on, to provide a set for each degree of freedom. Each joint is designed to be 
independently controlled for each set by control device 15. 



Table 1 



motion degree of freedom 


master motion direction 


slave motion direction 


base rotation 


101a 


101b 


shoulder joint rotation 


102a 


102b 


elbow joint rotation 


103a 


103b 


wrist joint (pitch) 


104a 


104b 


wrist joint (yaw) 


105a 


105b 


wrist joint (roll) 


106a 


106b 


gripper open/shut 


107a 


107b 



A camera arm 4 is linked at a first end thereof to the shoulder joint 10b of the slave, and 
the other end has a television camera 6 mounted thereon. On-site conditions, including 
the slave arm, are captured by the television camera 6 through the zoom lens 7, and the 
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images are displayed on a monitor television 5 situated in proximity to operator 9. The 
operator 9 manipulates master manipulator la while viewing the images, to control the 
slave manipulator lb. 

Control devices for each joint are substantially the same for each set, so 
hereinbelow the joints of one set shall be described by way of example. 

Fig. 1 is a control mechanism for joints of one set. A force sensor 22a, motor 20a, 

and position sensor 21a are linked to a master joint shaft 16a so as to interlock therewith. 

Similarly, a force sensor 22b, motor 20b, and position sensor 21b are linked to a slave 

joint shaft 16b so as to interlock therevsdth. A zoom lens 7 changes its zoom factor M by 

means of a zoom conmiand signal Cz from a switch 37. Current zoom factor M is sensed 

by a sensor 23. A zoom factor signal M and a master current position signal Pm are input 

to a computing unit 29. The output of computing unit 29 is PM/(kiM), which is input to a 

slave position target setting unit 3 1 . The output Psr of slave position target setting unit 

31 is input to a computing unit 24 together with the output signal Ps of slave position 

sensor 21b. The output Psr-Ps of computing unit 24 is input to slave motor 20b. A slave 

current position signal Ps and zoom factor signal M are input to a computing unit 28, 

giving kiMPs. The output signal kiMPs of computing unit 28 is input to computing unit 

26 together with the master current position signal Pm, giving (kiMPs-Pivi). This output 

signal is input to a switch 34. The output signal Fs from slave force sensor 22b is input to 

a master force command setting unit 35 together with zoom factor signal M, giving a 

master force command value Fmr. A master current force signal Fm, which is the output 

signal of master force sensor 22a, is input to a subtracter 25 together with the master force 

command value Fmr, giving the output (Fmr-Fm). Together with the output signal of 

switch 34, signal (Fmr-Fm) is input to master motor 20a via an adder 27. Zoom signal M 

is input to a gate generating circuit 30, whereby the foUovdng gate signal Gi is output. 

{0(dM/dt = 0) 
Gi= {l(dM/dt^O) 

Gate signal Gi is input to slave position target setting unit 3 1 and switch 34. The 
output signal Psr of slave position target setting unit 3 1 is as follows, depending on gate 
signal Gi. 
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{PM/(k,M) (G,=0) 
PsR= {PsRo(Gi = l) 

Here, Psro is PM/(kiM) immediately prior to Gi changing from 0 to 1 . 

The output signal Pme of switch 34 becomes 

{0 (Gi = 0) 
Pme= {kiMPs (Gi-1). 

The master force command value Fmr which is the output of switch 35 is as 

follows, depending on a mode command signal Cf from switch 36 and the gate signal Gi 

from the gate generating circuit 30. 

0 (Gi = 1) 
Fmr= {k2Fs (G, = 0,Cf = 0) 
{k2MFs (Gi = 0,Cf-1) 

Parameter Cf is a conraiand signal designating a control mode for force 
transmitted from the slave to the master, and is used to select whether the magnitude of 
transmitted force should change according to zoom factor. 

To sununarize the preceding, signals input to the master and slave motors 20a, 
20b depending on the state of zoom factor m and mode command signal Cf are indicated 
in Table 2. 

Table 2 



Cf 


dM/dt 


master motor signal 


slave motor signal 


0 


0 


k2Fs -Fm 


(PM/KiM)-Ps 


0 


5^0 


kiMPs -Pm 


PsRO 


1 


0 


kiMFs -Fm 


(PM/KiM)-Ps 


1 


^0 


kiMPs -Pm 


Psro 



The principle of operation is now described. The master and slave joints are 
controlled as follows, by the values of Cf and dm/dt (= dM/dt). 
A Cf = 0, dM/dt = 0 

A master joint is controlled so as to generate force equivalent to a multiple k2 of 
the force being applied to a slave joint from the outside. Positioning of the slave joint is 
controlled according to movement of the master joint. However, the angle of motion 0mr 
of the master joint is multiplied by a factor of 0M/kiM for transmission to the slave, 
whereby the angle of motion of the slave as it follows the master joint is 
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esR = 9M/kiM ....(1) 

This state is illustrated in Fig. 3. It is assumed that the master joint [moves] from 
point Amr to point Bmr. In the absence of the circuitry taught in the present invention, 
the slave joint simply rotates from point Ai to point Bi by an angle 6si (=6m); on the 
monitor screen being viewed by the operator, [the joint appears to] move from point An to 
point Bn by an angle MGsi. Accordingly, even where the operator moves the master joint 
only slightly, since the motion is [magnified] by factor M when displayed on the monitor 
screen, moving the slave joint a slight amount on the monitor screen requires moving the 
master joint even more slightly, by a factor of 1/M. This makes manipulation difficult, 
and places a considerable burden on the operator. 

According to the present embodiment, however, to move the slave joint by [an 
angle of] 0si on the monitor screen, it is sufficient to simply move the master joint by the 
same angle Gmr = 6si; the actual angle of movement Gsr of the slave joint at this time is 
0SR = 1/M0SI (= (1/M)0MR ), i.e. scaled down by a factor of 1/M. Accordingly, even 
where the operator has zoomed in on an image containing the slave manipulator on the 
monitor screen, it is a relatively simple matter to finely position the slave manipulator 
while viewing the slave manipulator on the monitor screen. 

Specifically, as shovm in Fig. 4, when the operator 9 grasps the handle portion 13a 
of manipulator la and moves the handle portion 13a by [a distance] Im parallel to the 
screen of the monitor television, [the slave manipulator appears to move by distance] li 
on the television screen, li is given by the following equation, where Is is the actual 
distance traveled by the slave manipulator. 

li=mls (2) 

Here, m is the magnification to the television monitor image relative to the actual 
image of the slave joint; with respect to the distance L (Fig. 2) between the television 
camera 6 and the gripper 13b slave manipulator lb, the relationship 

m = mo ' M ' L (3) 

holds. Since Is = Ps, (2) can be written as 
li = mo ' M • L • Ps. 

As will be apparent from Table 2, since position control is done such that Ps ^ (PM/kiM), 
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1, = (moL/ki)PM 
Further, since Pm = 1m, 

Ii/lM = moL/ki ....(4) 
In order to maintain a constant ratio of the distance 1m over which the master 
manipulator moves to the distance li over v^hich the slave manipulator [appears to] move 
on the television monitor, it is sufficient to perform control such that moL/ki is constant. 
Accordingly, the arrangement for position control of camera arm 4 is such that 

L = Lo (= a constant value) (5) 

Here, it is given by 

ki = (l,/(moLo-lM) ....(6). 
Li this embodiment, camera arm 4 is mounted on shoulder joint 10b, so a degree 
of freedom of 3 is acceptable, with a construction having the directions of movement 1 10 
-113 shown in Fig. 2. 

The master joint is controlled so as to generate force equivalent to the force Fs 
exerted on the slave joint from the outside, multiplied by a force gain k2. Here, the 
rigidity (spring constant k) of an object on which the slave manipulator is currently 
exerting force is given by the following equation, 
k = Fs/ ....(7) 
Here, APs denotes deformation resulting from application of force by the slave 
manipulator to the object. From Table 2, the master joint is force-controlled such that 
reaction force is generated by the master manipulator in the form Fm ^ k2Fs. Where the 
slave manipulator undergoes displacement of APs, from Table 2, the master manipulator 
will need to imdergo displacement of APm = kiMAPs. As a result, the rigidity 
transmitted to the manipulator is given by 

k3 = (Fm/APm) = (k2Fs/kiMAPs) = (ki/ki) • (kM) 

(8) 

Accordingly, where 

k2 = k, (9) 

is selected, rigidity transmitted to the master will be 1/M. Thus, even if the slave 
manipulator is handling a hard object, the operator controlling the master manipulator 
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will have the perception of handUng a soft object. If the object is vibrating, the vibration 
will be attenuated by a factor of 1/M when transmitted to the master manipulator, so that 
unnecessary information is not transmitted to the master manipulator. 
B CF-l,dm/dt = 0 

Depending on the task, it may be necessary to accurately transmit to the master 
actual rigidity on the slave side. In such instances, the Cf = 1, dm/dt = 0 mode is used. 
Specifically, since Fm ^ k2MFs, 

k3 = (Fm/APs) = (k2F2/kiAPs) = (ks/ki) k .... (10) 

Where kj/kx is selected, 

k3 = k ....(11). 
C Where zoom factor M is changing (dm/dt ?^ 0) 

Where the zoom factor M changes, the slave position command value Gsr changes 
according to Equation (1). This means that the slave manipulator will give rise to drift 
motion due to change in the zoom factor, regardless of the master manipulator. In modes 
A and B, it is necessary to take care to avoid the slave manipulator colliding with 
surrounding obstacles during change in the zoom factor. There is a risk that unnecessary 
zoom operation may result in the slave manipulator colliding with an obstacle, possibly 
breaking the chassis and the obstacle. Accordingly, when the zoom factor M is changing, 
calculations are performed such that the position command values Psr for all joints of the 
slave manipulator are held at the values Psro just prior to the change, and position control 
is performed such that slave joints are subsequently kept stationary in their current 
positions. At the same time, master joints are position controlled such that 

PM^kiMPs ....(12). 
If this position control were not executed, even at the same magnification with the master 
manipulator placed in the same joint attitude, the slave manipulator will not retum to the 
same location, so that the location cannot be reproduced. In this embodiment, position 
control represented by Equation (12) is executed by master joints, whereby position 
reproducibility between the master manipulator and slave manipulator is not impaired. 
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According to the embodiment described hereinabove, there is the advantage that 
master and slave manipulators can be controlled vsdthout being aware of changes in zoom 
ratio. 

Similar advantages could be obtained by means of another embodiment wherein a 
joystick is used in place of a master manipulator, by controlling the speed of the slave 
joints such that the ratio Vr/M of zoom factor M and the slave speed instruction value vr 
by the joystick is equal to the speed command value of the slave manipulator. 

As another variation, a 3-D television could be used in place of a television 
monitor. This variation has the advantage that finer positioning is possible. 

As yet another variation, similar advantages could be obtained by employing a 
method for enlarging/shrinking the monitor image by means of an image processing 
device, rather than a zoom lens, as the means for changing the zoom factor on the 
television screen. This variation has the advantage of obviating the need for a zoom lens, 
thus reducing the weight of the slave manipulator. Similar advantages could also be 
obtained by changing the distance L between slave gripper 13 and television camera 6, 
instead of an image processing device. 
(Effects of the Invention) 

According to the invention set forth herein, movement of the slave manipulator is 
[multiplied by] a factor of 1/M depending on the zoom factor M, so that the amount of 
movement in the manipulator image displayed on the monitor television is dependent 
solely upon the command value by the operator, irregardless of the zoom factor, thereby 
dramatically improving the ease of fine movement of the manipulator. 

Brief Description of the Drawings 

Fig. 1 is a block diagram showing an embodiment of a manipulator control device 
according to the invention; Fig. 2 is a schematic illustration of an exemplary manipulator 
system embodying the invention; Fig. 3 shows the relationship between master and slave 
angles of movement; and Fig. 4 shows the relationship between master and slave 
displacement on the monitor television. 
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1 ... manipulator, 4 ... camera arm, 5 ... monitor television, 6 ... television camera, 
7 ... zoom lens, 15 ... control device, 20 ... motor, 21 ... position sensor, 22 ... force 
sensor, 23 ... zoom factor sensor, 24 -29 ... computing units, 30 ... gate generating circuit, 
31 ... slave position target setting device, 34 ... switch, 35 .. master force command setting 
device, 36, 37 ... switches of control device 15 

Fig. 1 
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